<html>
  <head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <title>MediaSource Extensions</title>
    <link rel="stylesheet" href="video-working-draft.css">
    <link rel="stylesheet" href="main.css">
  </head>
  <body>
    <div class="head">
      <h1>MediaSource Extensions v0.3</h1>
      <h2>Draft Proposal</h2>
      <dl>
<dt>Editor:</dt>
<dd>Aaron Colwell, <a href="http://www.chromium.org/">Chromium</a> &lt;acolwell@chromium.org&gt;</dd>
</dl>
    </div>
    <h2>Status of this Document</h2>
    <p>This document is a draft specification proposal with no official status. Send comments to  <a href="mailto:acolwell@chromium.org">Aaron Colwell</a>. It is inappropriate to cite this document except as a work in progress.</p>

    <h2>Abstract</h2>
    <p>
      This proposal extends HTMLMediaElement to allow 
      JavaScript to generate media streams for playback. 
      Allowing JavaScript to generate streams facilitates a variety of use 
      cases like adaptive streaming and time shifting live streams.
    </p>

    <h2>Table of Contents</h2>

    <ul id="toc" style="list-style-type:none">
      <li><a href="#introduction">1. Introduction</a></li>
      <li>
<a href="#mediasource">2. Media Element Extensions</a>
      <li><a href="#event-summary">3. Event Summary</a></li>
      <li><a href="#algorithms">4. Algorithms</a></li>
        <ul style="list-style-type:none">
          <li><a href="#algorithms-attach">4.1 Attaching to a media element</a></li>
          <li><a href="#algorithms-detach">4.2 Detaching from a media element</a></li>
          <li><a href="#seeking">4.3 Seeking</a></li>
        </ul>
      </li>
      <li><a href="#examples">5. Examples</a></li>
      <li><a href="#revision-history">6. Revision History</a></li>
    </ul>

    <h2 id="introduction">1. Introduction</h2>
    <p>Historically &lt;audio&gt; and &lt;video&gt; have only allowed media playback from container file formats. Support for use cases like live have required
    hacks to the container format, like 0 size segments in WebM, or required the browser to support manifest formats like <a href="http://tools.ietf.org/html/draft-pantos-http-live-streaming-06">HLS</a>. Supporting new adaptive streaming proposals like <a href="http://www.3gpp.org/ftp/Specs/html-info/26247.htm">3GP-DASH</a> also require significant browser changes and the adaptation algorithms cannot be changed once the browser is released. The file container model also puts constraints on on-demand playback. Efficient seek support requires an index to be transfered to the browser which results in either delayed startup, or delays during the first seek. All of these issues stem from the fact that the browser is expected to manage all media fetching and do so based on the information contained in a single URL.</p>
    <p>This proposal allows JavaScript to control how media data is fetched. This will enable significant flexibility for media fetching without requiring browser changes. Manifest formats like HLS &amp; 3GP-DASH could be handled in JavaScript. Indexes for on-demand playback would not be needed because JavaScript could make an AJAX call to a web service to find an appropriate seek point. Adaptive streaming algorithms could be implemented and evolved over time without needing to upgrade the browser. Fetching media from multiple CDNs or splicing in media from the local file store could also be implemented without browser changes. Allowing JavaScript to control media stream construction opens many avenues for experimentation and innovation.</p>

    <h2 id="mediasource">2. Media Element Extensions</h2>
    <p>We extend HTML media elements to allow media data to be streamed into them from JavaScript.</p>
    <pre class="idl">
partial interface HTMLMediaElement {
  // URL passed to src attribute to enable the media source logic.
  readonly attribute [URL] DOMString <a href="#dom-webkitmediasourceurl">webkitMediaSourceURL</a>;

  bool <a href="#dom-webkitsourceappend">webkitSourceAppend</a>(in Uint8Array data);

  // end of stream status codes.
  const unsigned short <a href="#dom-eos_no_error">EOS_NO_ERROR</a> = 0;
  const unsigned short <a href="#dom-eos_network_err">EOS_NETWORK_ERR</a> = 1;
  const unsigned short <a href="#dom-eos_decode_err">EOS_DECODE_ERR</a> = 2;

  void <a href="#dom-webkitsourceendofstream">webkitSourceEndOfStream</a>(in unsigned short status);

  // states
  const unsigned short <a href="#dom-source_closed">SOURCE_CLOSED</a> = 0;
  const unsigned short <a href="#dom-source_open">SOURCE_OPEN</a> = 1;
  const unsigned short <a href="#dom-source_ended">SOURCE_ENDED</a> = 2;

  readonly attribute unsigned short <a href="#dom-webkitsourcestate">webkitSourceState</a>;
};

    </pre>

    <p>The <dfn id="dom-webkitmediasourceurl"><code>webkitMediaSourceURL</code></dfn> attribute returns the URL used to enable the MediaSource extension methods. To enable the MediaSource extensions on a media element, assign this URL to the <code><a href="http://www.w3.org/TR/html5/video.html#attr-media-src">src</a></code> attribute.
    </p>

    <p>The <dfn id="dom-webkitsourceappend"><code>webkitSourceAppend(data)</code></dfn> method must run the following steps:</p>
    <ol>
      <li>If the first argument is null or <var title="true">data</var>.byteLength is 0 then throw an <code><a href="http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#dom-domexception-invalid_access_err">INVALID_ACCESS_ERR</a></code> exception and abort these steps.</li>
      <li>If the <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> attribute is not in the <code><a href="#dom-source_open">SOURCE_OPEN</a></code> state then throw an <code><a href="http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#dom-domexception-invalid_state_err">INVALID_STATE_ERR</a></code> exception and abort these steps.</li>
      <li>Copy contents of <var title="true">data</var> into the media element's decode buffer.</li>
      <li>Return true.</li>
    </ol>

    <h5>End of stream status values:</h5>
    <dl>
      <dt>
<dfn id="dom-eos_no_error"><code>EOS_NO_ERROR</code></dfn> (numeric value 0)</dt>
      <dd>The stream ended normally without any errors.</dd>

      <dt>
<dfn id="dom-eos_network_err"><code>EOS_NETWORK_ERR</code></dfn> (numeric value 1)</dt>
      <dd>The stream ended prematurely because of a network error. If the JavaScript code fetching media data encounters a network error it should use this status code to terminate playback. This will cause the media element's error handling code to run and the <code>error</code> attribute to be set to <code><a href="http://www.w3.org/TR/html5/video.html#dom-mediaerror-media_err_network">MediaError.MEDIA_ERR_NETWORK</a></code>
</dd>

      <dt>
<dfn id="dom-eos_decode_err"><code>EOS_DECODE_ERR</code></dfn> (numeric value 2)</dt>
      <dd>The stream ended prematurely because there was an error while decoding the media data. If the JavaScript code fetching media data has problems parsing the data it should use this status code to terminate playback. This will cause the media element's error handling code to run and the <code>error</code> attribute to be set to <code><a href="http://www.w3.org/TR/html5/video.html#dom-mediaerror-media_err_decode">MediaError.MEDIA_ERR_DECODE</a></code>
</dd>
    </dl>

    <p>The <dfn id="dom-webkitsourceendofstream"><code>webkitSourceEndOfStream(status)</code></dfn> method must run the following steps:</p>
    <ol>
      <li>If the <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> attribute is not in the <code><a href="#dom-source_open">SOURCE_OPEN</a></code> state then throw an <code><a href="http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#dom-domexception-invalid_state_err">INVALID_STATE_ERR</a></code> exception and abort these steps.</li>
      <li>Change the <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> attribute's value to <code><a href="#dom-source_ended">SOURCE_ENDED</a></code>.</li>
      <dl class="switch">
          <dt>If <var title="true">status</var> is set to <code><a href="#dom-eos_no_error">EOS_NO_ERROR</a></code>
</dt>
          <dd>Notify the media engine that it now has all of the media data. Playback should continue until all the media passed in via <code><a href="#dom-webkitsourceappend">webkitSourceAppend()</a></code> has been played.</dd>
          <dt>If <var title="true">status</var> is set to <code><a href="#dom-eos_network_err">EOS_NETWORK_ERR</a></code>
</dt>
          <dd>Run the "If the connection is interrupted, causing the user agent to give up trying to fetch the resource" section of the <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a>
</dd>
          <dt>If <var title="true">status</var> is set to <code><a href="#dom-eos_decode_err">EOS_DECODE_ERR</a></code>
</dt>
          <dd>Run the "If the media data is corrupted" section of the <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a>
</dd>
          <dt>Otherwise</dt>
          <dd>Throw an <code><a href="http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#dom-domexception-invalid_access_err">INVALID_ACCESS_ERR</a></code> exception.</dd>
        </dl>
    </ol>

    <p>The <dfn id="dom-webkitsourcestate"><code>webkitSourceState</code></dfn> attribute represents the state of the source. It can have the following values:</p>
    <dl>
      <dt>
<dfn id="dom-source_closed"><code>SOURCE_CLOSED</code></dfn> (numeric value 0)</dt>
      <dd>Indicates the source is not currently attached to a media element.</dd>

      <dt>
<dfn id="dom-source_open"><code>SOURCE_OPEN</code></dfn> (numeric value 1)</dt>
      <dd>The source has been opened by a media element.</dd>

      <dt>
<dfn id="dom-source_ended"><code>SOURCE_ENDED</code></dfn> (numeric value 2)</dt>
      <dd>
<code><a href="#dom-webkitsourceendofstream">webkitSourceEndOfStream()</a></code> has been called on this source.</dd>
    </dl>
    <p>When the media element is created <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> must be set to <code><a href="#dom-source_closed">SOURCE_CLOSED</a></code> (0).
    </p>

    <h2 id="event-summary">3. Event Summary</h2>
    <table>
      <thead>
        <tr>
          <th>Event name</th>
          <th>Interface</th>
          <th>Dispatched when...</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td><dfn id="dom-webkitsourceopen"><code>webkitsourceopen</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the source transitions from <code><a href="#dom-source_closed">SOURCE_CLOSED</a></code> to <code><a href="#dom-source_open">SOURCE_OPEN</a></code> or from <code><a href="#dom-source_ended">SOURCE_ENDED</a></code> to <code><a href="#dom-source_open">SOURCE_OPEN</a></code>.</td>
        </tr>
        <tr>
          <td><dfn id="dom-webkitsourceended"><code>webkitsourceended</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the source transitions from <code><a href="#dom-source_open">SOURCE_OPEN</a></code> to <code><a href="#dom-source_ended">SOURCE_ENDED</a></code>.</td>
        </tr>
        <tr>
          <td><dfn id="dom-webkitsourceclose"><code>webkitsourceclose</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the source transitions from <code><a href="#dom-source_open">SOURCE_OPEN</a></code> or <code><a href="#dom-source_ended">SOURCE_ENDED</a></code> to <code><a href="#dom-source_ended">SOURCE_ENDED</a></code>.</td>
        </tr>
      </tbody>
    </table>

    <h2 id="algorithms">4. Algorithms</h2>
    <h3 id="algorithms-attach">4.1 Attaching to a media element</h3>
    <ol>
      <li>Set the <code><a href="http://www.w3.org/TR/html5/video.html#attr-media-src">src</a></code> attribute on a media element or the <code><a href="http://www.w3.org/TR/html5/video.html#attr-media-src">src</a></code> attribute on a &lt;source&gt; element associated with a media element to <code><a href="#dom-webkitmediasourceurl">webkitMediaSourceURL</a></code>
</li>
      <li>
<p>When the media element attempts the <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a> with the URL from <code><a href="#dom-webkitmediasourceurl">webkitMediaSourceURL</a></code> it will take one of the following actions:</p>
        <dl class="switch">
          <dt>If <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> is NOT set to <code><a href="#dom-source_closed">SOURCE_CLOSED</a></code>
</dt>
          <dd>Abort media element's <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a> and run the steps to report a <code>MEDIA_ERR_SRC_NOT_SUPPORTED</code> error.</dd>
          <dt>Otherwise</dt>
          <dd>
            <ol>
              <li>Set <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> attribute to <code><a href="#dom-source_open">SOURCE_OPEN</a></code>
</li>
              <li>Fire a simple event named <code><a href="#dom-webkitsourceopen">webkitsourceopen</a></code>.</li>
              <li>Allow <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a> to progress based on data passed in via <code><a href="#dom-webkitsourceappend">webkitSourceAppend()</a></code>
</li>
            </ol>
          </dd>
        </dl>
      </li>
    </ol>

    <h3 id="algorithms-detach">4.2 Detaching from a media element</h3>
    <p>The following steps are run in any case where the media element is going to transition to <code><a href="http://www.w3.org/TR/html5/video.html#dom-media-network_empty">NETWORK_EMPTY</a></code> and fire an <code><a href="http://www.w3.org/TR/html5/video.html#event-mediacontroller-emptied">emptied</a></code> event. These steps should be run right before the transition.</p>
    <ol>
      <li>Set <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> attribute to <code><a href="#dom-source_closed">SOURCE_CLOSED</a></code>
</li>
      <li>Fire a simple event named <code><a href="#dom-webkitsourceclose">webkitsourceclose</a></code>.</li>
    </ol>


    <h3 id="seeking">4.3 Seeking</h3>
    <ol>
      <li>The media element <code><a href="http://www.w3.org/TR/html5/video.html#dom-media-seeking">seeking algorithm</a></code> starts and has reached the stage where it is about to fire the <code><a href="http://www.w3.org/TR/html5/video.html#event-media-seeking">seeking</a></code> event.</li>
      <li>If the <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> attribute is set to <code><a href="#dom-source_ended">SOURCE_ENDED</a></code>, then the media element sets the <code><a href="#dom-webkitsourcestate">webkitSourceState</a></code> attribute to <code><a href="#dom-source_open">SOURCE_OPEN</a></code>
</li>
      <li>The media element flushes all data from previous <code><a href="#dom-webkitsourceappend">webkitSourceAppend()</a></code> calls.</li>
      <li>The media element <code><a href="http://www.w3.org/TR/html5/video.html#dom-media-seeking">seeking algorithm</a></code> fires the <code><a href="http://www.w3.org/TR/html5/video.html#event-media-seeking">seeking</a></code> event and will proceed once data is passed to the media element via <code><a href="#dom-webkitsourceappend">webkitSourceAppend()</a></code>.</li>
    </ol>
    <p class="note">
      This seeking algorithm assumes that the data passed to <code><a href="#dom-webkitsourceappend">webkitSourceAppend()</a></code> is packetized media with timestamps (ie Ogg pages or WebM clusters). This allows the "new playback position" to be determined by the timestamps in the data. If we want to support formats that are not packetized, like WAV files, then we will need a different mechanism to communicate the "new playback position" when a seek occurs.
    </p>

    <h2 id="examples">5. Examples</h2>
    <p>Example use of MediaSource</p>
    <div class="block">
      <div class="blockContent">
        <pre class="code">
&lt;script&gt;
  function onOpen(e) {
    var video = e.target;

    var headers = GetStreamHeaders();

    if (headers == null) {
      // Error fetching headers. Signal end of stream with an error.
      video.webkitSourceEndOfStream(HTMLMediaElement.EOS_NETWORK_ERR);
    }

    // Append the stream headers (ie WebM Header, Info, and Tracks elements)
    video.webkitSourceAppend(headers);

    // Append some initial media data.
    video.webkitSourceAppend(GetNextCluster());
  }

  function onSeeking(e) {
    var video = e.target;

    // Notify the cluster loading code to start fetching data at the
    // new playback position.
    SeekToClusterAt(video.currentTime);

    // Append clusters from the new playback position.
    video.webkitSourceAppend(GetNextCluster());
    video.webkitSourceAppend(GetNextCluster());
  }

  function onProgress(e) {
    var video = e.target;

    if (video.webkitSourceState == video.SOURCE_ENDED)
      return;

    // If we have run out of stream data, then signal end of stream.
    if (!HaveMoreClusters()) {
      video.webkitSourceEndOfStream(HTMLMediaElement.EOS_NO_ERROR);
      return;
    }

    video.webkitSourceAppend(GetNextCluster());
  }

  var video = document.getElementById('v');
  video.addEventListener('webkitsourceopen', onOpen);
  video.addEventListener('seeking', onSeeking);
  video.addEventListener('progress', onProgress);
&lt;/script&gt;

&lt;video id="v" autoplay&gt; &lt;/video&gt;

&lt;script&gt;
  var video = document.getElementById('v');
  video.src = video.webkitMediaSourceURL;
&lt;/script&gt;
        </pre>
      </div>
    </div>

    <h2 id="revision-history">6. Revision History</h2>
    <table>
      <thead>
        <tr>
          <th>Version</th>
          <th>Comment</th>
        </tr>
      </thead>
      <tbody>
	<tr>
          <td><a href="http://html5-mediasource-api.googlecode.com/svn/tags/0.3/draft-spec/mediasource-draft-spec.html">0.3</a></td>
          <td>Minor text updates.</td>
        </tr>
        <tr>
          <td><a href="http://html5-mediasource-api.googlecode.com/svn/tags/0.2/draft-spec/mediasource-draft-spec.html">0.2</a></td>
          <td>Updates to reflect initial WebKit implementation.</td>
        </tr>
        <tr>
          <td><a href="http://html5-mediasource-api.googlecode.com/svn/tags/0.1/draft-spec/mediasource-draft-spec.html">0.1</a></td>
          <td>Initial Proposal</td>
        </tr>
      </tbody>  
    </table>
  </body>
</html>
